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ABSTRACT 

This report presents the development and results of a 
1-year foreign language module program of 15-weeks duration for 
university students in the arts, commerce, science, and engineering. 
The project's central concern was to develop and test a general model 
for the delivery of foreign language teaching to students of other 
disciplines. The program included audio-video learning units, 
satellite television facilities, and computers. Modules were designed 
to engage students in immediate, active language use and piactice of 
the target language (German or French) to provide a solid foundation 
in basic speaking, listening, reading, and writing skills; classes 
focused on broadly defined domains of communicative activity. 
Assessment took place in weeks 5 and 6 and involved 113 students. 
Results indicate that students completing the modules attained a 
satisfactory level of language competence as measured by deployment 
of communicative skills and general underlying proficiency. External 
evaluators rated the project as successful as well. Special funding 
permitted the modules to be continued for a second year. Appendixes 
include a list of related publications, an outline of the French 
language module for students of health sciences, a summary of 
expenditures (in British pounds), band descriptors and rating sheet, 
and a correlational analysis of test components. (NAV) 



ED 386 923 

AUTHOR 
TITLE 



INSTITUTION 

PUB DATE 
NOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



Reproductions supplied by EDRS are the best that can be made 
" from the original document. 

i: •!! i! *■>!*■>! it i! -it ■:<*■;:■>( it it it -it it -it -it 



Report on the ESF-Funded Project to Consolidate 
and Develop Foreign Language Modules for Students 
of other Disciplines Implemented by the Centre for 
Language and Communication Studies 
1 October 1993-30 September 1994 



PERMISSION TO REPRODUCE THIS '-.-.^H^.Iil'^.^Ji^l'f. 



MATERIAL HAS BEEN GRANTED BY 

_ v> C- . , 





EDKCATlOMAl RESOURCES INPOHMATIO!J 
CENTER (cRlCl 



□ [/I'loi rl\int)Gs h.lvc h(-f"-» in.^oo Ki 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) - 




University of Dublin ♦ Trinity College 



Report on the ESF-funded project to 
consolidate and develop foreign language modules for 
students of other disciplines 
implemented by the 
Centre for Language and Communication Studies 
1 October 1993-30 September 1994 



Contents 



1 introduction 

David Little, 
Director, CLCS 



Implementation 

Etna Ushioda 
Project co-ordinator 



3 External evaluation 

Edith Esch and Anny King 
Language Centre 
University of Cambridge 



4 Conclusion 

David Little 



Appendices 



1.1 The Centre for Language and 
Communication Studies 

1.2 Background to CLCS's language 
modules project 

1.3 The launch of CLCS's language 
modules project 

2.1 Consolidation of existing language 
modules 

2.2 Recruitment of students to take the 
new modules 

2.3 Rates of participation 

2.4 Design of the new modules 

2.5 Design of assessment procedures 

2.6 Assessment results 

3.1 Introduction 

3.2 Design of the pilot project 

3.3 The pilot project in the context of 
language learning provision in higher 
education in Europe 

3.4 Management of the project 

3.5 Evaluative approach adopted for the project 

3.6 Recommendations for the future 
Notes 

4.1 Response to the external evaluators' 
recommendations 

4.2 Provision of language modules in 1994-5 

4.3 Research dimensions 

4.4 The future 



ERIC 



Applied linguistic research in CLCS 

French module for students of 

Health Sciences 

Summary of expenditure 

Band descriptors and rating sheet 

Correlational analysis of tost components 



7 
9 
11 

13 
13 



14 
15 
16 
16 
17 

18 
IK 
19 
20 

22 

24 
25 
26 
28 



BEST COPY AVAILABLE 



1 Introduction 



David Little 



1 .1 The Centre for Language 
and Communication 
Studies 

The Centre for Lnngunge nnd Communication 
Studies (CLCS) wns established in 1978 as a 
special development associated with the 
commissioning of Trinity College's new Arts 
and Social Sciences Building. 

Fron-i the beginning CLCS has had a threefold 
function: (i) to provide technical support for 
language teaching and learning; (ii) to engage in 
theoretical and applied linguistic research; and 
(iii) to develop an appropriate teaching role. 

For a number of years CLCS lias provided 
courses in theoretical and applied linguistics at 
undergraduate and postgraduate levels. In 
Trinity term 1990 the University Council made 
CLCS responsible for co-ordinating the 
pro\'ision of language teaching for students not 
taking a foreign language as part of their degree 
programme. The project that is the focus of this 
report represents CLCS's first involvement in 
language teaching on a large scale. 

Over the past fifteen years applied linguistic 
research in CLCS has focussed on many 
different aspects of language learning and 
language teaching, including: self-access and 
self-instruction; learner autonomy; intcractix'e 
video for language learning; the definition of 
language teaching syllabuses; the de\'elopment 
of language learning materials; pedagogical 
grammar; the development of the second 
language mental lexicon in learners at uni^'ersity 
level; the ago factor in language learning; thi 
role of the mother tongue in language learning 
(Appendix 1, pp.22f, below, lists n selection of 
relevant publications). 

At e\'ery stage CLCS's language modules 
project has been informed by one or another 
strand of relevant applied linguistic research. 



and as it has proceeded it has generated 
research dimensions of its own. It is worth 
emphasizing that this is by no means a common 
occurrence. Even in universities language 
teaching of the kind described in this report is 
all too often treated as a service provision which 
neither needs nor deserves the benefits of 
theoretical and empirical research. 

1 .2 Background to CLCS's 
language modules project 

In Trinity term 1993 the Higher Education 
Authority announced an ESP-funded pilot 
project for the restructuring of primary degrees 
and invited proposals from the universities. One 
of the three measures to be implemented was 
the introduction of foreign language modules 
for students in the Faculties of Arts, Commerce, 
Science and Engineering. 

In its submission to the HEA Trinity College 
proposed (i) to consolidate language modules 
already offeied by enhancing the facilities 
available to support them, and (ii) to establish 
new language modules. 

As regards consolidation, by the academic year 
1992-3 the following courses stood to benefit 
from the proposed project: 

• in the Faculty of Business, Economic and 
Social Studies, parth' integrated two-\'ear 
courses in French (non-beginners) and 
German (non-beginners), taken by students 
in their junior and senior freshman (first and 
second) years; 

• in the Faculty of F.ngmeering and Systems 
Sciences, partly integrated two-year courses 
in French (non-beginners) and German 
(beginners and non-beginners), taken by 
students in their senior freshman and junior 
sophister (second and third) years; 

• in the Faculty of Science, a non-integrated 
two-vear course in German (beginners and 



non-beginners), taken by students in their 
senior freshman and junior sophister years. 

As regards new modules, the project's central 
concern was to develop and test a general model 
for the delivery of foreign language teaching to 
students of other disciplines, hi other words, it 
was to bo a major concern of the project to lay 
down firm foundations for the future 
development of the undergraduate curriculum 
in this area. The proposal submitted to the HH A 
included the following new modules: 

• French for non-beginners in Science; 

• French for non-beginners in Arts; 

• German tor beginners in Arts; 

• Gorman for non-beginners in Arts. 

A self-access programme in German for Science 
students was among the language modules 
already provided in Trinity College (see abo\'e), 
and for this reason the College's proposal did 
not include German for Science students among 
tlie new modules. Howe\'er, scarcely any 
Science students enrolled in the existing 
programme at the beginning of the academic 
year 1993-4, so the following new modules were 
added to the project: 

• German for beginners in Science; 

• German for non-beginners in Science. 

The fact that only French and German were 
provided for is a reflection of the dominance of 
these two languages in the school curriculum 
and in relation to El^SMUS student exchange 
programmes. The latter dominance emerged 
clearly in a survey of foreign language needs 
that was undertaken in Michaelmas term 1990 
by the Director of CLCS in consultation with 
faculty deans. The situation had not changed 
when the exercise was repeated in Hilary term 
1993. 



Students in the Faculty of Health Sciences were 
not included in the population targeted by the 
HEA's pilot project. However, in Michaelmas 
term 1993 the Faculty asked CLCS to provide a 
F'rench module for non-beginners in Hilary and 
Trinity terms 1994 in order to support 
ERASMUS exchanges with institutions in 
France. The Faculty financed the module, which 
was organized within the general framework 
established for the ESF-funded modules. For 
purposes of comparison, a brief report on rates 
of participation and assessment results is 
provided in Appendix 2, p. 24 below. 



1 .3 The launch of CLCS's 

language modules project 

The proposal that Trinity College submitted to 
the HEA was based on a budget for 1993-4 of 
£168,000 and the assumption that additional 
accommodation could be made awiilable to 
house new technical installations. In the event, 
the HEA awarded the project a grant of £100,000 
and no additional accommodation could be 
made a\'nilable. 

The project was launched with the approval of 
the University Council early in Michaelmas term 
1993. Ema Ushioda was appointed project co- 
ordinator. Marc Gallagher was appointed 
pedagogical adviser, and Dr Edith Esch and 
Anny King of the Language Centre, Uni\'ersitv 
of Cambridge, were appointed external 
evaluators. 

The funding period of the project was from 1 
October 1993 to 30 September 1994. Appendix 3, 
p. 25 below, provides a summary of expenditure. 



2 Implementation Emauswoda 



2.1 Consolidation of existing 
language modules 

Before the project began CLCS offered the 
following technical support for language 
learning: 

• an audio-active-comparative language 
laboratory of 40 booths, used almost 
exclusively for self-accoss learning; 

• an audio-activc-comparntive language 
laboratory of 20 booths, used almost 
exclusively for class teaching; 

• a network of 12 computers, used almost 
exclusively for class teaching; 

• 3 individual satellite television viewing 
facilities offering broadcasts in Trench, 
German and Russian. 

Trinity College's proposal to the HI: A included 
the provision of 20 individual video playback 
and satellite television viewing facilities, to be 
installed in a room of their own. But since no 
additional accommodation was available, it was 
necessary to expand and enhance technical 
support for language learning within existing 
accommodation. Obviously, it was important to 
find space for new facilities and functions 
without disrupting well-established patterns of 
class teaching and self-access learning. At the 
same time, the decision was taken to expand not 
only individual video playback and satellite 
television viewing but also computer facilities, 
since the project offered an opportunity to open 
the network to self-access learning. 

The following measures were taken priiir to the 
launch of the new modules in Hilary term 1 W4: 
• The 12 booths arranged in pairs along the 
right-hand wall of the large language 
laborator)' were remo\-ed. l'i\'e of them were 
added to the main block of 28 booths on the 
left-hand side of the lahoratorv, giving a total 
of 33 booths for self-access learning; li\ e 
more were added to the small language 
laboratory, increasing its capacity to 25; and 



the remaining two booths were put into 
storage. 

• 15 individual video playback and satellite 
television viewing facilities were installed 
along the right-hand wall of the large 
language laboratory. In addition, 
adjustments were made to the distribution 
svstem for satellite television signals to allow 
a maximum of ten signals to be deli\'ered to 
the large language laboratory via a single 
cable. This not only simplified the 
installation of the new facilities but opened 
up the possibility of making broadcasts in 
languages other than i-rench, German and 
Russian available at relatively low cost in 
due course. 

• The existing Nimbus computer network was 
reduced from 12 to 10 machines and a 
second network of 10 PCs was installed in 
the same room. Purpose-built furniture was 
bought for the two networks. 

• Since any expansion of self-access activity 
raises new security problems, surveillance 
cameras were installed in the two language 
laboratories and the computer room, and a 
monitor in the main CI-CS office. 

• Finally, the main CLCS office was compleleh' 
reorganized to accommodate the project co- 
ordinator, the pedagogical adviser and the 
additional language learning resources 
generated by the project. 

2.2 Recruitment of students to 
take the new modules 

The new modules were scheduled to begin in 
the first week of Hilary lecture term 1994, in 
order to allow sufficient time for planning, 
publicity and recruitment during Michaelmas 
lecture term. 

I he module-^ could not be acconimod.ited 
within the normal lecture timetable, so it was 
agreed with the rcspecti\'e faculty deans that 
they would run on Saturday mornings Irom 1(1 
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n.m. to 12 noon for students in Science, nnd on 
Wednesday evenings from 7 to 9 p.m. for 
students in Arts. 

It wns nlso agreed thnt the modules would be 
offered to nil junior freshmen in Science, nnd to 
.ill junior nnd senior freshmen in Arts. An 
importnnt considerntion wns the fnct thnt the 
opportunity to undcrtnke study nbrond is 
gcnernlly offered to students in their second or 
third yenr. \n the finnl stnges of recruitment, 
however, plncus remnining were offered to the 
rest of the undergrndunte populntion in Science 
nnd Arts — i.e., senior freshmen nnd sophister 
(third nnd fourth yenr) students in Science, nnd 
sophister students in Arts. 

For students in Arts, it wns ngreed thnt priority 
should be given to those not studying n foreign 
Inngunge ns n degree subject, though within the 
Two-subject Moderntorship there seems no . 
good renson in principle why n student 
combining Germnn with history should be 
prevented from reviving her French with n view 
to studying history nt n French-spenking 
university. 

The modules were publicized in two wnys. For 
junior freshmen in Science, n genernl meeting 
wns held in Week 5 of Michnolmns term (on 12 
November 1993), nnd wns nttended by nn 
estimnted 150 out of n possible totnl of 227 
students. For junior nnd senior freshmen in Arts, 
n two-dny informntion nnd recruitment session 
(18 nnd 19 November) wns held. This wns 
publicized by menns of posters which hends of 
depnrtment were nsked to displny on 
depnrtmentnl noticebonrds. 

Applicntions were processed on n "first come 
first served" bnsis, the minimum requirement 
for those npplying for non-beginners' modules 
being n LenvLng Certificnte (or equivnlent) 
qunlificntion in their chosen Inngunge. 

The notionnl mnximum number of students to 
be ndmitted to ench module wns 50, which 
yielded n mnximum of 300 students ncross the 
six modules — npproximntely n qunrter of the 
totnl freshmnn populntion in Science nnd Arts. 
In prnctice, ndmissions wore hnndled flexibly to 
bnlnnce the numbers between over-subscribed 
nnd under-subscribed modules. There wns n 
particularly high demand for the French 
modules (see Table 2.1, p. 6 below). 



Applications were also accepted from 14 
students who did not belong to that part of the 
population targeted by the project, but who had 
expressed a strong interest in the modules and 
had applied in the early stages of recruitment. 
These included students from the Faculty of 
Engineering and Systems Sciences and the 
Faculty of Business, Economic and Social 
Studies, as well as a small number of 
postgraduate students. For the purposes of the 
project statistics, however, these students were 
treated as supernumeraries and not included in 
the monitoring of participation figures. 



2.3 Rates of participation 

The new modules began in the first week of 
Hilary lecture term 1994, and ran for a total of 
15 weeks to the end of Trinity lecture term. In 
order to monitor rates of participation, regular 
attendance records were kept for the weekly 
contact sessions. In the early weeks of Hilary 
Term, the rate of withdrawal was to some extent 
matched by late admissions and participation by 
a handful of unregistered students in the contact 
sessions. As places became available, these 
students were officially admitted to the 
modules. 

In Table 2.1 and Figure 2.1, p.6 below, the rate of 
weekly participation is expressed as a 
percentage of the total number of students 
enrolled for each module. This includes 
confirmed and unconfirmed withdrawals, as 
well as week-by-week increases through late 
admission of applicants during Hilary Term. 
These increases explain why the total 
recruitment figure for all six modules exceeds 
the initial maximum of 300 students referred to 
in 2.2. It should be added that minor 
adjustments in the numbers enrolled are also 
due to transferrals by a few students from one 
module to another (e.g., from German for 
beginners to German for non-beginners or from 
Saturday mornings to Wednesday evenings). 

Although the participation le\'els showed a 
steady decline from the initial high rates of 
attendance, the general picture which emerges 
scorns on the whole \'ery positi\'e when put into 
porspecli\'e. It was expected that the decline 



Module 


Initial 
total 
recruitment 


Number 
attending 
first session 


Final 
total 
recruitment 


Number 
completing 
module 


French non-beginners 
in Arts (Wednesday) 


70 Arts 
2 Science 


58 (81%) 


76 


22 (29%) 


Trench non-beginners 
in Science (Saturday) 


78 Science 
1 Arts 


70 (89';;>) 


85 


32 (38';;>) 


German non-beginners 
in Arts (Wednesday) 


32 Arts 


30 (94";{>) 


38 


11 (29%) 


German non-begiiiners 
in Science (Saturday) 


44 Science 
2 Arts 


41 (89':;,) 


47 


29 (62':;>) 


German beginners 
in Arts (Wednesday) 


45 Arts 
1 Science 


32 (70"/.,) 


47 


13 (287;,) 


■ German beginners 
in Science (Saturday) 


26 Science 


22 (85';.;.) 


27 


6 (22';'o) 



Table 21 

Rates of participation in French and German modules 



Total Student Participation 



— Wednesday 



Attendance 




10% ■■ 

0% \ 1 1 1 1 1 1 > « 1 1 ' — ' — ' — ' 

HT HT HT HT HT HT HT HT HT TT TT TT TT TT TT 
123456789123456 

Weel<s (HT:Hilary Term, TT:Trinity Term) 



_J 

Figure 2.1 



would be steeper in the early \ve(?ks of the 
modules, and thi*^ proved to be the ciso. As 
Figure 2.1 shows, the rate of altendanco ti iided 
to level off about the middle of Hilary term. A 
further drop in participation was expected in 
Trinity term, with the onset of examination 



pressures from the students' main subjects of 
study. While participation rates were lower in 
Trinity than in Hilary term, attendance during 
Trinity term nevertheless remained steady. 
Indeed, it showed a slight upward trend 
towards the end of the modules. In all, 113 



students underwent assessment in Weeks 5 and 
6 of Trinity term. 

As Figure 2.1 clearly shows, a higher rate of 
participation was maintained for the Saturday 
morning than for the Wednesday evening 
modules, even though attendance on Saturdays 
was sometimes affected by the scheduling of 
field-trips. Two possible reasons for this 
difference in participation levels may be 
inferred. Feedback from students who withdrew 
from the modules indicated that the pressure of 
a full timetable was the most common cause, 
cited by over 70% of those who provided 
explanations for dropping out. With the normal 
college lecture timetable stretching from 9 a.m. 
to 6 p.m. and sometimes 7 p.m. for students in 
Arts, it seems probable that the strain of a 
fiu'ther two-hour midweek evening session 
proved too great for many students who 
embarked on the Wednesday c\'ening modules. 
In contrast, physical or mental fatigue was less 
likely to be a contributing negative factor on 
Saturday rnornings, once students had 
committed themselves to devoting this period of 
their week to language study. 

It should also be noted that students in Science 
(Saturday morning modules) were gi\'en the 
incentive that extra percentage points would be 
added lo their end-of-year examination results if 
they did well in the language modules. 

The differing rates of participation on 
Wednesday evenings and Saturday mornings 
suggest that two important issues would need to 
be addressed if the language modules were to 
be fully integrated into the primary degree 
structure: (i) a reduction in students' workload 
to compensate for the extra demands of 
language learning; and (ii) integration of the 
grade achie\'ed into students' end-of-year 
examination result. 



2.4 Design of the new modules 

{•ach module in\'ol\'ed students in two contact 
hours per week. In addition they were 
encouraged to devote a furlht-r hmireach wwk 
to sclf-ncccss learning. 

Learners' needs— ikr.nise of time constraints it 
was not possible to undertake a needs analysis 
sur\'ey among ^he target student population. 



Instead, students' needs were defined in terms 
of the general rationale underlying the proposed 
restructuring of primary degrees. In the short 
term this entailed the development of basic 
communication skills for the purposes of study, 
travel or work experience abroad during 
students' undergraduate years. In the longer 
term it entailed the development of practical 
language skills to enhance students' academic 
qualifications, vocational prospects and 
potential for future mobility. Added value and 
future mobility were highlighted in the publicity 
for the modules. 

Course design— In accordance with the short- 
term needs specified above, the modules were 
designed to engage students in immediate active 
use and practice of their target language so as to 
provide them with a solid foundation in basic 
speaking, listening, reading and writing skills. 
One of the reasons why so many language 
teaching programmes fail to make a long-term 
impact on their learners is that they do nothing 
to make them users of their target language. 
Perhaps the easiest way to ensure that language 
learners become language users is to employ the 
target language as the medium of classroom 
management and instruction. This procedure 
has the added advantage that it increases the 
amount of target language "input" that learners 
receive and interactively stimulates the 
"digestion" of that input. 

Classes focussod ni"- broadly defined domains of 
communicative activity so as to provide a 
meaningful context within which to de\'elop, 
recycle and extend the students' range of 
functions, notions, lexis and grammar, and 
promote productive and receptive skills. 

For the beginners' German modules, these 
domains of communicative acti\ ily 
corresponded to the practical demands of 
survival in a German-speaking en\'ironment, 
such as basic greetings and social interactions, 
tra\'el, obtaining food and drink, and finding 
accommodation, and gradually progressed 
towards dealing with bureaucracy and taking an 
interest in cultural issues and current affairs. 

For the non-beginners' modules, the domains of 
CMmniunicative activity moved through three 
phases: (i) consolidation of basic communicative 
skills for e\'eryday sur\'i\Ml and social needs; (ii) 
refinement of expressive and interactive skills 
for effective invoK-ement in social and studv 
en\'ironments abroad; (iii) de\'olopment of 
spoken and written discursive skills and 



extended listening skills for the purposes of 
study abroad, using academically appropriate 
subject matter. 

Contact sessions were initially structured so as 
to provide students with thematic exposition 
and organizational guidelines from the 
language teacher (30 minutes), followed by 
rotations between, on the one hand, small-group 
work with native speaker student assistants (30 
minutes), and on the other, te.icher-supervised 
self-access work using the language laboratory, 
computer laboratory, satellite television and 
video playback facilities (30 minutes). Sessions 
ended with a final integration and consolidation 
of the tasks and material co\'ered with the 
teacher (30 minutes). 

As things turned out, by the later weeks of 
Hilary term both teachers and students felt that 
a longer opening session could be more usefully 
exploited for task-based student acti\'ities 
deriving from the initial exposition. In addition, 
a number of more academically-focussed short 
talks were arranged for the final half-hour 
period during these weeks. These talks on 
science and arts topics were gi\'en by F-rench- 
and German-speaking assistants as appropriate. 
Again it was felt that such talks could be more 
effectively deployed as expository material for 



listening input, leading to productive tasks 
integrating speaking and writing skills (e.g., 
asking questions, taking notes, summarizing). 
Tor these reasons, a modification was made to 
the two-hour structure of the contact session in 
Trinity term: a full hour was devoted to initial 
exposition leading to \'arious task-based 
acti\'ities; in the second hour, students di\'idcd 
into small groups for work with nati\'c-spcaker 
student assistants and self-access work. 

Meeting learners' long-term needs— If a 

language teaching programme is to be of long- 
term benefit to students it must help them to 
improve the efficiency of their learning. In other 
words, it must seek to foster learner autonom)', 
encouraging students to accept responsibility for 
their learning and to develop an ever more 
finely tuned awareness of language and 
language learning. With a \'iew in particular to 
students' long-term needs — \'iz., the 
enhancement of \'Ocational prospects and future 
mobility — it was desirable to encourage the 
development of a capacity for continued 
learning on an independent basis through the 
remaining undergraduate years. It was hoped 
that this would be achieved through the 
integration of self-access learning in the 
modules. 
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Self-access language learning facilities enable 
learners to make use of resources (e.g., printed 
material, audio and video tapes) and facilities 
(e.g., a language laboratory) on an individual 
basis and at times that fit in with their other 
commitments and activities. Ideally, self-access 
systems are supported by an advisory service 
that helps learners to determine their general 
learning strategy, select learning materials 
appropriate to their needs and interests, explore 
and refine their preferred methods of learning, 
and develop a capacity for self-assessment. 
Within the particular time constraints of the 
two-term modules in their pilot year, it was not 
possible to give self-access learning the high 
profile originally intended, although the overall 
rates of usage of CLCS's self-access facilities 
were encouraging (see Tigure 2.2, p. 8 abo\'e). 

2.5 Design of assessment 
procedures 

Tor the purposes of formal student assessment, 
two types of test were used — (i) a pcncil-and- 
paper te;it to measure students' general 
underlying control of the target language 
system; and (ii) a communicative test to assess 
their ability to deploy their language skills 
interactively. 

The pencil-and-paper test— For the pencil- 
and-paper test two instruments were chosen: a 
short dictation and n battery of four C-tcsts. A 
substantial body of empirical research has 
shown that both instrimients are highly reliable 
indicators of global language proficiency. 

The dictation was read by a native speaker and 
prerecorded for transmission via individual 
headphones in the language laboratory. It 
consisted of a lOO-word text, road through at 
normal speed, then read phrase by phrase with 
each phrase repeated once, and finally read 
through once again at normal speed. The 
dictation was scored at iwo levels: one mark for 
each correctly transcribed word; half a mark for 
each phonologically approximate word. 

The C-tcst is a modified form of the clo/.e test in 
which parts ol words rather than whole words 
are deleted. The beginning of the text is left 
intact to pro\'ide a contextuali/.ing lead-in, 
following which e\'ery second word has its 
second iialf deleted. A battorv of loiu' texts w.i^ 

o 
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chosen for each module, each text containing 25 
deletions. The C-tcsts were scored for correct 
responses only. 

In terms of cost-effectiveness, dictation and the 
C-test have the advantages (i) that they are oasv 
to administer and to score, and (ii) thai the same 
established battery of unseen texts can be used 
from year to year. 

All five texts used in the pencil-and-paper test 
were authentic in origin, although for test 
purposes it was necessary to edit them slightly. 
The texts lay within the thematic range of 
material treated in the module in question, and 
were selected on the basis of an intuitive 
judgement that they were likely to present the 
more able students taking the module with no 
serious comprehension problems. 

The communicative test— The communicative 
test was concei\'ed as a short group presentation 
(5-6 minutes) in the appropriate language on a 
chosen topic, for performance in front of an 
audience made up of fellow students and four 
independent assessors. This form of assessment 
offered three advantages: economy; focus on 
deployment of communicative skills; and 
integration with the design and deli\'ery of the 
modules. 

The group format of the test provides a method 
of assessing the oral skills of a large n'.imbcr of 
students which is more efficient and cost- 
effective than individual inter\'iews. The format 
of the presentation provides a natural vehicle 
for the deployment of acquired language skills 
in a meaningful and interactive context. 
Individual questioning of students following the 
presentation ensures that the assessment 
includes a judgement of spontaneous 
communicative ability. 

The performance context has high face validity, 
since students could conceivably be called upon 
to perform similar types of tasks in their second 
language in a study or work environment 
abroad. Also, because preparation of the group 
presentations is integrated into the o\'erall 
design and delivery of the modules, there is a 
direct correspondence between learning 
objectives and assessment of student 
performance. During the weeks leading up to 
the presentations, the group sessions with 
nati\'e-speaker student assistants were de\ >ted 
to their preparation. During this period also, 
attention was given to the rhetorical skills that 




students need in order to make an effective 
formal presentation. 

The group assessment rating-scales were 
designed to incorporate a grade for earh 
student's performance and an ox'erall grade for 
collaborative group perlormance. Individual 
student performances were scored on a 5-band 
scale for each of three elements: phonology, 
fluency and grammatical control. 

Group performances were given a single score 
on a 10-band scale for the quality with which 
content, vocabulary, language use, presentation 
and rhetorical skills were integrated in the 
collaborative execution of the task. A system of 
penalties was devised for group performances 
which failed to adhere to two principal 
criteria — viz., that the presentation must be 
interactive, and that students must not read 
from scripts. Groups failing to fulfil one of the 
two criteria were scored on bands 1-8 only, 
while groups failing to fulfil both criteria were 
scored on bands 1-6 only. 

The scales employed in the assessment of both 
individual and group performances were 
defined by appropriate verbal band descriptors. 
In adi ^tion, the projected maximum global level 
of attainment for students was defined (see 
Appendix 4, pp.26f. below, for non-beginners 
rating-scales with full band descriptors and a 
sample rating-sheet). 

It should be noted that the communicative test 
and the rating-scales were piloted at the end of 
Hilary term. The piloting entailed a statistical 
analysis of inter-rater reliability and detailed 
consultation with the external assessors (see also 
3.5 below). 

Validity and reliability — Por this pilot phase of 
the modules, it was thought particularly 
important to assess the validity and reliability of 
the combined testing instrument. A statistical 
analysis was therefore carried out of (i) the 
degree of correlation between student results 
across the separate test components and (ii) the 
degree of inter-rater reliability in the 
communicatix'e test conducted in Trinity term. 

As regards the modules for non-beginners, the 
analysis of student results rcx'caled moderate 
posilix c correlations between the communica- 
tive test and the pcncil-and-paper lest, 
suggesting that the assessment instruments 
chosen provided comparable measures of 



language proficiency, but that they were 
tapping different aspects of this proficiency. The 
statistical analysis thus supported the dual 
objectives of the two types of test, which were to 
assess ability to deploy communicatix'e 
language skills on the one hand, and underlying 
knowledge of the target language system on the 
other. 

The pattern of correlations between the 
communicative and pencil-and-paper test 
components was much weaker in the analysis of 
student results in the German beginners' 
modules. This was interpreted as a function of 
the type of language teaching to which the 
students had been primarily exposed. A strong 
emphasis on the development of basic oral and 
aural comm.unicatix'e skills meant that measures 
of communicative production were much less 
likely to be strongly correlateci with more 
system-referenced measures of target language 
knowledge. 

As expected, for all modules the analysis 
rex'ealed a strong positive correlation between 
the parallel components of the pencil-and-paper 
test. 

The analysis of inter-rater reliability in the 
communicative test revealed a \'ery high level of 
agreement across the four independent raters in 
both French and German modules. Reliability 
coefficients of 0.89 or greater were obtained in 
the analysis of total test scores across raters, 
thereby sustaining confidence in the design of 
the testing instrument and rating-scales, and 
confirming the implementation of this mode of 
assessment in future modules, in conjunction 
with the pencil-nnd-paper test. A summary of 
the statistical analysis is given in Appendix 5, 
p. 28 below. 

Calculating grades — The ox'erall grade for each 
student was calculated on the basis of 
performance in both the pencil-and-paper test 
and the communicatix'e test. In view of the more 
direct correspondence between the group 
presentation and the content and learning 
objectives of the modules, the communicative 
and pencil-and-paper tests wore weighted in the 
ratio of 3 to 2. Within the peucil-and-paper test 
the weighting of the C-tcst battery and dictation 
was in the ratio of 2 to 1, to reflect the 
proportions of time allowed for the romplelion 
of these components (-10 minulf' .md 20 minutes 
respeclix'oly). 
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Fail 


Fail 


III 


II.2 


II. 1 


I 


207r - 29% 


30% - 39% 


Am - A'-yy, 


50% - 59% 


UY/r - b9% 


li)'/, - 80% 


Hand level 1 


1.5 2 


2.5 3 


3.5 4 


4.5 5 


Kcnv score 


M) 40 


50 M) 


70 HO 


90 100 



Table 2.2 

Transposition of raw scores to classes of pass and fail (Arts students) 



The use of a 5-bnnd scale for the communicative 
test scores was designed to facilitate the 
transposition of students' total raw scores (i.e., 
combined communicative and pencil-and-paper 
test scores) to the four classes of pass and two 
classes of fail employed in the College's 
examination system. Table 2.2 shows the 
relation between the marks and classes used in 
Arts examinations and raw scores distributed 
across our five bandf ^n individual student's 
rau' score was transposed to the Arts system of 
marks and classes by a simple formula: multiply 
by 0.6 and add 20. 

In the Faculty of Science examinations are 
marked using the full percentage scale. 
Accordingly, for students in Science, the range 
of percentage grades attainable in the first class 
was expanded from 70-80'>!> to 70-100'>!>. The 
lower end of the scale would have been 
expanded downwards iiad any Science students 
failed the end-of-module examination. 

Full integration of the modules within the 
primary degree curriculum was not attempted 
during this pilot year. However, the University 
Council determined that the trades achieved bv 
students in their language module "should be 
presented as part of their annual examination 



result". In addition to the overall grade, 
students were gi\'en a more detailed breakdown 
of their attainment levels across the \'arious 
skills. This information should help those 
students who intend to engage in further 
language l'_arning. 



2.6 Assessment results 



As noted above, assessment took place in Weeks 
5 and 6 of Trinity Term, and 113 students 
participated in the assessment process. The 
results are summarized in Table 2.3 below, 
which indicates the range of student perfor- 
mance in each module, and Figure 2.3, p. 12 
below, which conflates the results in a single 
histogram. 

The results indicate that all students completing 
the modules attained a satisfactory level of 
language competence as measured in terms of 

(i) their deployment of communicative skills and 

(ii) their general underlying proficiency. Taken 
collectively, 62 (55%) of the 113 students 
completing the modules achieved a II. 2 grade, 
while 49 (43%) achieved a II. 1 grade. 



Module 


Student nos 


Average mark 


Highest mark 


Lowest mark 


I-runch for non-beginners in Arts 


20 


60% 


67% 


.=;2'>;. 


French for non-be);inncTS in Science 


34 


57'n, 


65"/., 


Sl% 


Gcrm.in (or iKin-bi;ginncrs in Arts 


12 




69% 


55"/.. 


Gcrmnn for nun-ht'i^innurs in Science 


28 


60"/,. 


70% 


■m., 


Germnn for bc);inncrs in .^rts 


13 


65';;. 


66% 


5\".,. 


Ciermnn for Lx');inners in Science 


b 




63% 


50"., 



Tabic 2.3 — Assessment results (Arts and Science) 
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3 External 
evaluation 



Edith Esch and 
Anny King 



3.1 Introduction 



From the beginning of the project, it \s'as agreed 
thnt ns external evnluntors we should fulfil n 
consultnti\'e role during its developmentnl 
stages, ns well ns offer n qualitntive commentnry 
on the project overnll in terms of its design, 
mnnngement, outcomes and implicntions for the 
future. We were fully aware that the project's 
central concern was to establish a viable 
framework for the provision of foreign language 
teaching for non-languago students, within 
particular budgetary and time constraints. 

Our evaluation is based on observations made 
during our visits to Trinity College, our 
consultations with the project team, and the 
regular reports and student assessment results 
submitted to us. The evaluative report has five 
sections: design of the project; the project's 
significance in the wider context of higher 
education in Europe; project management; the 
e\'aluative approach adopted; and 
lecommendations for the future. 



3.2 Design of the pilot project 



Has the programme achieved its primary aim of 
developing and testing a model for the delivery 
of foreign language teaching to students of other 
disciplines? In our view, the answer to this 
question is an emphatic "yes". We would like to 
draw attention in particular to the following 
aspects of the project design. 

Language learning needs analysis and 
syllabus design — An effective and highly 
successful module design was implemented, 
shaped by flexibility and a firmly learner- 
centred approach within the framework of a 
specified syllabus content. One of the problems 



of designing a foreign language syllabus for 
students In non-language degree programmes is 
the difficulty of catering for varying le\'els of 
proficiency, as well as wide-ranging needs and 
interests. Students may quickly lose motivation 
if they do not percei\'e the language programme 
content to be appropriate or relevant to their 
needs. In these language modules, this problem 
was resoK'ed through a flexible learner-centred 
approach to the selection of topics and the 
negotiation of learning activities and materials. 
The great range of interests displayed by 
student groups in their assessment presentations 
was strong evidence that such flexibility was 
important, since it would have been impossible 
to predict and designate appropriate topics and 
activities in advance. 

The approach was clearly successful, both in 
maintaining high numbers of students across the 
modules despite the study pressures of their 
other subjects, and in producing excellent 
results among a multi-level mixed-ability 
population of non-specialist students. 
Outstanding success was e\'ident in particular in 
the non-beginner modules, where the range of 
proficiency levels on entry differentiated 
between students more than in the beginner 
modules. 

A very useful tool for the future would be a 
detailed account of the actual procedures 
whereby students' needs and the syllabus were 
brought together, topics and materials were 
selected, and learning activities were negotiated. 
It is clear that the constraints of time and 
student numbers during this pilot phase made 
such detailed documentation impossible. 
Nevertheless, given the success of the project 
this year, an information resource of this kind 
would be most valuable. 

The modular concept— One of the 

fundamental concepts of the Council of 
F.urope's modern languages projects is the 
design of learning units or "blocks" which are 
relevant to the communicative needs of 
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particular cntcgorics of Icnrncrs. This concept 
wns developed in the 1970s by Trim et nl.' It is 
appropriate to refer to it here becnuse the CLCS 
language modules demonstrated the 
pedagogical, methodological and institutional 
advantages that it offers. 

Pedagogical advantages— From the point of 
view of (he students, the modules offer learning 
targets which are meaningful, realistic and 
attainable within a relatively short period of 
time. By its very nature the module structure is 
far more likely to increase access to foreign 
languages than other programme designs. The 
enthusiastic response to the modules at the 
beginning of the year testifies to this. 

Methodological advantages— Improving 
access to foreign languages "generally" is not 
sufficient. If it were, spending money on 
equipment would be enough. In our \'iew tl.u 
project showed particular strength in that it 
gave students access to a language learning 
method and to principles for managing their 
own learning. This was partly the result of the 
high degree of in-built flexibility for private 
study and the learner support provided. Above 
all, however, access to effective learning 
methods was gained through the task-based 
group work activities. These encouraged 
creativity and collaborative learning, produced 
some remarkable results, and integrated the 
processes of evaluation and peer-assessment. - 
This achievement is reflected in the success with 
which students worked together at the 
assessment stage. Its implications are far- 
reaching, since the acquired self-management 
skills and cooperative learning techniques arc 
transferable to other domains, and will 
hopefully be deployed throughout students' 
lives. 

Institutional advantages— The main 
institutional advantage of the modular structure 
of the pilot language programme is its 
flexibility. The modules are modifiable and 
transportable. Y-ov example, they could readily 
be adapted to a semester rather than trimester 
structure if necessary, and they could lend 
themselves to shi^rt intensi\'e programmes 
compressed over a few days or weeks, although 
such schemes tend to have fewer long-term 
benefits than more extended courses. The 
modules could also bo adapted by other 
institutions, or tailored to languages other than 
French or German. 



A second institutional advantage lies in the 
likely integration of module development with 
the predicted rapid expansion of technical 
support in the future. For example, one can see 
the potential benefits of creating a readily 
accessible learner support system through the 
use of electronic mail, and of developing a 
flexible network of communication links of this 
kind with French-and German-speaking 
students in other institutions in Furope. 
Technical expansion through European high- 
speed information networks would also make 
available a range of CALL (computer-assisted 
language learning) and multimedia packages, 
such as those produced in the United Kingdom 
in the framework of the Teaching and Learning 
Technology Programme. Such materials could 
be used as learning/teaching resources and 
integrated into the modules, thus making it 
possible to offer language modules to larger 
numbers of students in Ireland. It is conceivable 
that such developments in the use of 
information technology will lead to its partial 
deployment in assessment procedures also.-' 

A third major advantage is that the language 
modules could lend themselves to the Council of 
I'urope system of "credits" — i.e., qualifications 
which can be combined in a flexible fashion. The 
National Vocational Qualifications (NVQs) are 
recognized by employers in the United 
Kingdom, and may soon become valid 
throughout Furope under the Council of Europe 
system of credits. Validating the language 
modules in terms of NVQs would require time, 
research and effort, but would offer significant 
potential benefits and attractions to students, 
who would be able to "sell" their linguistic skills 
more easily on the national and F.uropean 
market alike. Such a project for the future seems 
highly desirable. 

3.3 The pilot project in the 
context of language 
learning in higher 
education in Europe 

Non-compulsory character of the 
modules — i-rom our own expeiience at the 
University of Cambridge, we approve of the 
decision to offer the language modules on a 
voluntary enrolment basis. At Cambridge, non- 
compulsory French and German programmes 
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were introduced in the new four-year 
Engineering Tripos. The programmes included 
classes comparable in many ways to the 
language modules at TCD, and were offered 
during the first two years of the course. Wc 
found that student take-up of the language 
programmes was \'ery high, and that althougii 
class attendance did show some decline through 
the year, about 60"/<) of students in the first two 
years were in\'olvcd. 

How 'e\'er, one of the worrying problems in 
sustaining a non-compulsory scheme of tiiis 
kind at third level is that it may not be funded 
properly. Lack of adequate funding has ob\'ious 
repercussions on provision for staff recruitment 
and training, with the ultimate higiily negative 
scenario in which foreign language programmes 
for non-language degree students are poorlv 
taught and reduced to second-rate status in 
higher education. 

'ihe project was particularly successful in 
effccti\'ely deploying the guidance of two 
experienced teachers, the skills and talents of a 
small team of nnti\'e speaker language tutors, 
and the enthusiasm and motivation of a large 
number of l-rench and German student 
assistants. It would be regrettable if I ck of 
funding meant that this combined team 
approach could not be sustained in future 
module development. It is not possible, for 
example, that native-speaker student assistants 
on their own, e\-en with appropriate short-term 
training, could replace experienced and 
qualified teachers. This aspect needs to be 
carefully monitored and the issue of funding 
addressed. 

Returning to the question of voluntary student 
enrolment, it seems that making foreign 
language learning compulsory may be 
counterproductive, in the short term at least. 
Potential employers are likely to respond more 
fa\'ourably to the evidence of real enthusiasm, 
effort and dedication shown by those who ha\'e 
participated \'oluntarily in language learning 
programmes, rather than to evidence of 
opportunism and the pre-packaged "flawnir of 
the day" degree. On this front, Irish students 
seem to have an advantage over their United 
Kingdom counterparts because the bilingual 
context in which they have been brought up 
appears to have instilled positi\'e attitudes to 
foreign language learning. Although they 
mostly share the same first language, the 
attitudinal pattern seems to be different among 
English students, according to the VoKi/y 



Europeans iii 19S7 survey conducted by the 
European Commission.'* 

Integration into specialists' degrees — The 

language modules could be tailored to provide 
an access route for students aiming to specialize 
in European course options leading to Euro- 
careers. In the Cambridge course mentioned 
above, a European option which forms part of 
the new four-year engineering degree is 
available from the third year onwards. Students 
taking this option have to work on engineering 
projects through the medium of Trench and/or 
C .'man and are assessed in these languages in 
.leir final examination. Making foreign 
language skills an entry requirement for 
students taking such courses would deny a large 
number of students access to these particular 
courses. Moreover, although all educated 
Individ'. -lis ideally need to bo able to survive in 
at least two other European languages, it is trise 
that only a proportion of students ' vill actually 
go for Euro-careers — all the more reason why 
access to courses with a European option should 
not be restricted at college entry level. The 
provision of language modules, on the other 
hand, gi\'es students who ha\'e special 
motivation or talent for languages the chance to 
prepare themseh-es to take a special option of 
this kind in their third or final year. 

Choice of French and German — The decision 
to offer modules in these particular languages 
makes sense not only for historical and cultural 
reasons, but more importantly for linguistic: 
reasons. Knowledge of French gives access to 
Romance languages such as Spanish, 
Portuguese, Italian and Catalan, and even to 
some extent Modern Greek. Knowledge of 
German of course gives access to Dutch, 
Luxemburgish and to a number of dialects 
spoken in Switzerland, Belgium and Prance. In 
thi s way, both languages offer direct access 
routes to a majority of the Indo-European 
languages spoken in the European Union. Por 
students who already ha\'e a Gaelic language, 
knowledge of French/German thus gi\-es an 
added bonus in the context of the European 
Union. 



3.4 Management of the project 

The management of the project was \'ery good 
indeed. A timetable was established from the 
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beginning, and the overall design of the project 
was very clearly stated so that there were no 
ambiguities. As evaluators, wo felt that we were 
given all the information we required, that 
everything was explicit and carefully planned, 
and that what we were asked to do met with our 
expectations of our role. Two points need a 
special mention for excellence. 

Firstly, the log-book system used to keep us 
informed of week-by-week project development 
was implemented in a very effective way. Its 
narrative aspect provided easy and multiple 
"entry points" which made it possible for us as 
evaluators to move into the project quickly and 
effectively. 

Secondly, the group assessment rating-grids 
were particularly well thought-out in terms of 
presentation and layout, as well as content. They 
were easy to understand, clear and explicit. This 
made it possible for any person not involved in 
the project to understand the structure and main 
categories for the implementation of the 
assessment instrument. We feel that CLCS 
should provide a "project management tool-kit" 
(on paper and diskette) to support and guide 
other institutions of higher education involved 
in similar projects. 

3.5 Evaluative approach 
adopted for the project 

We believe that the approach to project 
evaluation used here is to be recommended as n 
model of the genre. We certainly hope that it 
will be built upon and generalized in the 
profession. 

We find that the process of formative as well as 
sumniative evaluation is an excellent approach. 
It saves an enormous amount of time and turns 
a so-called objective "judgement" into a 
cooperative search between respected 
colleagues for the best solution in a given 
context. We do not wish to imply that our 
evaluation was somehow "cosmetic", howe\'er. 
Evaluation developed rather as a constructive 
process whereby any problems or criticisms wo 
raised could be discussed and dealt with 
immediately in a positi\'o cooperative fashion. 

This constructive process was best seen in the 
way in which commentary and feedback 



following the piloting of the group task 
assessment in Hilary term led to fine-tuning of 
the rating-grids and clarification of ta.sk criteria 
for the Trinity term assessment. In this way we 
believe we were able to ensure that our 
evaluation had maximum positive impact on 
project development while minimizing the face- 
threatening potential that our evaluative role 
might impose. 

In the context of language programme 
development of this kind, we are convinced of 
the mutual benefits of such evaluative 
procedures both for the institution undergoing 
evaluation and also for the evaluators, who 
learn from the process as well. We are currently 
leading a Teaching and Learning Technology 
Programme (TLTP) for the development of 
materials for language learning for Engineers 
and Scientists.^ We were therefore able to 
apprise the CLCS team of our own findings, 
while they could alert us to particular types of 
common problems which we had not foreseen. 
In other words, project evaluation was definitely 
a valuable exercise for both institutions and 
countries. 

3.6 Recommendations for the 
future 

1 The project should lead to a consolidated 
programme of language modules. 

2 A second-stage pilot should be conducted to 
include: 

(a) a detailed study of the modulo content 
and of the range of procedures used to 
match needs and syllabus. 

(b) a training module for all staff in-.'olvod. 

(c) a study of the way the modules could bo 
related to the NVQ scheme. 

3 The consolidated programme should be 
generalized to include other disciplines 
and/or languages if required. 

4 CLCS should plan to make information on 
the modules and their assessment available 
to other universities by means of advanced 
networking, such as Suporjanct, between 
universities. Problems of infrastructure are 
crucial at this juncture if duplication is to bo 
avoided. 



5 CLCS should produce n project mnnngcmcnt 
tool-kit for the mnnngement of similar 
langunge progrnmmes in other third-level 
institutions. This should include in particular 
an assessment procedure pack with all the 
rating-grids and explanations of what thev 
do, how to use them, etc. 

6 A promotional video should be produced of 
the group tasks performed b}' the students in 
Health Sciences, which could be included in 
the pack mentioned under 5, and used to 
train tutors and assistants, etc. Other \ ;deo- 
recordings should be made to train staff in 
the use of the assessment rating-grids. 



Notes 



1 S\/stciiis DciVlopiuciit in Atiult Lnii^na^c 
Learning, Council for Cultural Cooperation, 
Council of Europe, Strasbourg, 1973. 

2 Anny King's comment on the TCD medical 
students' performance was: "the result of 
non-specialist language teaching at its best". 



Students were using their specialist 
knowledge while being entertaining and 
imaginative in the second language. 
Moreo\'er, there was evidence of the 
educational \'alue of the exercise. The team 
effort had made students think about 
effectiv e communication in a cooperative 
fashion: they had been using each others' 
special talents to best effect for the group. 

3 Cf. publications of the Teaching and 
Learning Technology Programme (TLTP) 
and ALTER (Assessment of Learning 
through Technology for Efficiency and 
■Rigour) published by the UK Universities' 
Staff Development Unit (USDU) and the 
Uni\-ersities of Kent and Leeds. 

-1 Yoiiii;:; Eiiwpcniis ui 19S7, Commission of the 
European Communities, Brussels, 1988. 

5 TLTP Project CKS33, a Cambridge-led 

consortium for the production of multimedi 
packages in French and German for 
Engineers and Scientists. The Cambridge 
Language Centre is in partnership with the 
Uni\'ersities of Kent and Southampton for 
the production and with the Uni\'ersitics of 
Surrey and Durham for the e\'aluation. 
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4 Conclusion 



David Little 



4.1 Response to the external 
evaluators' 
recommendations 



The CLCS project team accepts the 
recommendations made by the external 
assessors in 3.6 above. 

The measures reported in section 4.2 below 
constitute acceptance of recommendation 1; 
while the three parts of recommendation 2 have 
been built into the organizational framework for 
1994-5. 

In the course of 1994-5 we plan to conduct a 
survey to establish what language learning 
needs exist in Trinity College beyond those 
covered by the modules wc have developed to 
date (recommendation 3). Previous surveys 
have indicated that the demand for languages 
other than French and German is likely to be 
relatively low (see 1.2 above). If this finding is 
confirmed, it will ha\'e obvious implications for 
the ways in which these needs should be met. 

Wo agree that it would be appropriate to make 
information on the modules and their 
assessment available to other 'universities 
(recommendation 4). As a first step in this 
direction, the present report will be distributed 
as widely as possible. Depending on the 
outcome of the Information Systems Policy 
Development exercise that Trinity College has 
been conducting during the academic year 
1993-4, it may be possible in the fairly near 
future to make information about the modules 
available via advanced networking, as the 
external assessors recommend. 

In the coirrso of 1994-5 wc plan to produce a 
project management tool-kit (recommendation 
5) and to make video recordings for use in the 
training of teachers, assistants and assessors 
(recommendation 6). 



4.2 Provision of language 
modules in 1994-5 



Funding — In its original invitation to the 
universities to submit proposals for funding, the 
Higher Education Authority emphasized the 
pilot nature of the 1993-4 projects. Nevertheless, 
at various stages in the academic year 1993-4 
the College was given to understand that 
funding would again be made available to 
support foreign language modules in 1994-5. It 
therefore came as an unwelcome shock when 
the HEA informed the College in July 1994 that 
no further funding would be available. 

The success of the pilot language modules had 
been such that it was unthinkable to the CLCS 
project team that they should be abandoned 
without further ado. In any case, the need to 
pro\'ide foreign language learning opportunities 
for students is not diminished by funding 
difficulties. 

Portunately it has proved possible to offer (i) the 
modules that were piloted in 1993-4 to a new 
generation of students in 1994-5, but o\'er three 
terms rather than two, and (ii) second-year 
modules to all students who successfully 
completed the pilot modules. CLCS has been 
able to draw on its own research and non-pay 
fundmg to provide the organizational basis for 
the modules, while the faculties in\'olved have 
agreed to meet the cost of hourly-paid teaching. 
Material support costs (learning materials, 
printing, stationery, etc.) will be met by 
requiring each student who enrols for a 
language module in 1994-5 to pay a levy of £20. 

It must be emphasized that these funding 
arrangements arc an aii lioc response to an 
emergency, and that they entail the use of 
resources that will not be available in future 
years. They are thus no more than a very 
temporary solution to a problem that the 
College must soh'o in the course of 1994-5 if it 
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wishes to provide foreign language modules in 
the long term. 

Organizational arrangements — An 

information sheet advertising the 1994-5 
modules has been sent to all incoming jimior 
freshmen in Arts, and similar pubHcity materials 
will be disseminated to incoming junior 
freshmen in other faculties during Freshers' 
Week (3-7 October 1994). Applications will be 
processed in the first two weeks of Michaelmas 
term, and the modules will begin in Week 3. As 
in 1993-4, preference will be given to 
applications from junior freshmen in Science 
and to applications from junior and senior 
freshmen in Arts. But as in 1993-4 we expect to 
be able to accommodate a small number of 
undergraduate students in other years as well as 
a few postgraduate students. 

Student induction — ^Thc first two weeks of the 
1994-5 modules will be given over to student 
induction. Students will be provided with 
information about the processes involved in 
foreign language learning and introduced to the 
self-access resources and facilities. Workshop 
activities will be used to explore appropriate 
approaches to learning, including the keeping of 
a regular log-book as a record of what work has 
been done, a compilation of what has to be 
learnt, and an assessment of how well learning 
is progressing. 

Induction of teachers and ERASMUS 
student assistants — Early in Michaelmas term 
applications will be sought from ERASMUS 
students who wish to act as language assistants. 
All applicants will be screened for their 
suitability — in particular they will be asked to 
make a brief presentation in French or German 
as appropriate. 

There will be formal meetings of teachers and 
ERASMUS students at the beginning and end of 
each term, and regular briefing and feedback 
sessions during term. 

Learning materials and resources for self- 
access — ^The core learning materials for both 
new and continuing students have been 
prepared, including a student handbook for 
each term. The handbook contains course 
materials, advice on language learning in 
general and the use of sclf-ncccss resources in 
particular, and templates and guidelines fur 
students' log-books. 



In the course of the year particular attention will 
be given to the development of resources to 
support the self-access component of the 
modules. Work in this area will benefit from 
exploratory visits made in 1993-4 to Paris and 
Frankfurt (Marc Gallagher), and to Wida 
Software, London (David Little, Ema Ushioda, 
Marc Gallagher). Provision will be made for the 
use of CD-ROM materials in CLCS's computer 
room. 

In March 1994 David Little and Ema Ushioda 
visited the Media Center for Language and 
Culture, University of Amsterdam, which has 
begun to develop a computerized catalogue of 
self-access language learning materials. Work 
was subsequently begun on the de\'elopment of 
a computerized catalogue for CLCS's materials, 
including a guide to their use. This project will 
continue in 1994-5. 



4.3 Research dimensions 



Section 1.1 of this report noted that the design 
and implementation of the pilot modules was 
able to draw on a strong research base in CLCS. 
This meant that the pilot modules rapidly 
generated a research dimension of their own, 
which will be continued in 1994-5 in five areas: 
assessment, interactive video for self-access 
language learning, learner autonomy, learner 
coimselling, and the use of information systems 
and electronic communication in foreign 
language learning. 

Assessment— At the Third CERCLES 
Conference, University of Hull, 23-25 
September 1994, David Little and Ema Ushioda 
gave a paper reporting on the development and 
validation of the assessment instruments used at 
the end of the pilot modules, with particular 
reference to the group presentations. In 1994-5 
these assessment instruments will be further 
developed to include a test of students' writing 
competence at the end of the second-year 
modules. 

Interactive video for self-access— For a 

number of years CLCS has been working 
experimentally with interactive vid"o for 
language learning in self-access. At EUROCALl, 
1994 in Karlsruhe, 15-17 September, David 
Little gave an invited keynote paper that 
elaborated the theoretical foundations of this 
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work, explored some dntn elicited from 
students, nnd set out n development plnn for the 
future. In 1994-5 CLCS intends to refine its 
interactive video system further, with n view to 
giving it n central role in the self-access 
component of the language modules from 1995- 
6 onwards. 

Learner autonomy — The fuller use of self- 
access resources will be influenced by David 
Little's continuing work on learner autonomy, 
which in 1993-4 included a study commissioned 
by the Council of Europe's modern languages 
project, two invited articles for international 
journals, and an invited keynote paper at an 
international conference on learner autonomy in 
Hong Kong. Pursuing a related line of research. 
Marc Gallagher gave a paper on learner 
autonomy at the Third CERCLIZS Conference, 
Uni\'ersity of Hull, 23-25 September. 

Learner counselling — A new research 
dimension is planned for 1994-5, focussing on 
the language learning process and learner 
autonomy. The intention is to de\'elop a 
succession of instruments to elicit data from 
students at various stages in the academic year, 
and to analyse the data thus elicited in such a 
way as to provide individual students with 
guidance on the direction of their further 
language learning. The instruments will include 
a questionnaire on students' language 
background, a language aptitude test, measures 
of student attitudes and motivation, a C-test 
battery, and learner log-books. 

This dimension of the project is closely related 
to the consultancy work David Little and Lmn 
Ushioda ha\'e undertaken for the Centre for 
Modern Languages at the Open University, 
Milton Keynes. 

Information systems and electronic 
communication— hi the course of 1994-5 the 
project team will explore the feasibility of 
establishing an international research project to 
look into ways of exploiting electronic 
communication and electronic databases to 
support language learning. In particular, wo 
hope that it may be possible to establish a 
triangular link between ourselves and 
universities in Trance nnd Germany, so that 
Irish learners of Trench and Cierman can be put 
in touch with l-rcnch- and Gennnn-speaking 
peers. 



4.4 The future 



At a formal institutional level Trinity College 
has been seriously imerested in providing 
language learning opportunities for the 
generality of students at least since Trinity tern-, 
1990, when (as noted in the introduction to this 
report — see 1.1 above) the University Council 
made CLCS responsible for the co-ordination of 
efforts in this area. 

The principal obstacle to the implementation of 
the proposals that were brought forward in 
Michaelmas term 1990 was lack of funding. This 
obstacle was temporarily overcome in 1993 by 
the sudden a\'ailability of special funding for 
the pilot project which is the subject of this 
report. As we move none too securely into the 
second year of the modules, two large issues 
have to be confronted: integration and long- 
term funding. 

Integration — Integration can be approached in 
two ways. On the one hand foreign language 
modules can become integral parts of the 
curriculum in the sense that they are cither 
compulsory for all students or else options that 
become compulsory for students who choose 
them. The obvious advantage of this approach is 
that there is no significant drop-out, which 
means that the organization of teaching can be 
finely calculated to be as cost-effective as 
possible. The disadvantage is that foreign 
language learning becomes one more subject of 
study, so that the provision of foreign language 
modules cannot reasonably be described as 
giving added \'aiuc. 

On the other hand, foreign language modules 
can be integrated in the weaker sense that they 
are designed to address students' needs in 
relation to their main academic discipline and 
related career options. The disadvantage of this 
approach is that there will always be a 
significant rate of attrition early in the academic 
year. The ad\'antage — which should not be 
lightly dismissed (see the report of the external 
evaluators, 3.3 abo\'e) — is that students who 
follow an optional programme of language 
learning to its conclusion mostly do so because 
their learning is successful. This helps to explain 
the fact that all students wlio completed the 
1993-4 pilot modules passed the end-of-year 
assessment; it also helps to explain why classes 
were so strongly cohesive and enjoyable. 
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Long-term funding— In Hilary term 1994 the 
question of the integration of foreign language 
modules was discussed by several Trinity 
College committees responsible for courses and 
their development. Each committee quickly 
came to the same conclusion: that the discussion 
was meaningless in the absence of guaranteed 
funding to support foreign language modules in 
the future. 

As this report shows, a great deal of progress 
has been made in a \'ery short space of time. 
What is more, that progress has been not simply 
in the pro\'ision of "service" language teaching, 
but in the beginnings of a solution to the 
problem of foreign language modules for all 
that started from a strong research base and has 
itself generated further research. This is b\' no 



means a widespread phenomenon, but it is 
surely one with which any imi\'ersity should be 
pleased. 

As noted in 4.2 abo\-o, the foreign language 
modules that were piloted in 1993-4 will be 
offered again in 1994-5, and second-year 
modules will be offered to students who 
successfully completed the pilot modules. This 
has been possible thanks to special financial 
circumstances that have been created in part by 
the very fact of the special funding made 
available in 1993-4. However, it will not be 
possible to continue the modules beyond 1994-5 
without the provision of at least some long-term 
funding security. How that is to be achieved 
must now be addressed as a matter of urgency. 
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Appendix 1 



Applied linguistic research in CLCS: 
selected list of publications 



Self-access and self-instruction 



Devitt, S. M., D. G. Little, S. P. 6 Conchuir nnd 
D. M. Cinglcton, 1982-3: "Learning Irish 
with Aiwif ;'s An':". CLCS Occnsionnl Paper 
No.6. 

Little, D. (^d.), 1989: Sc!f-Access Systems for 

LMiigim^c Lenriuu^. With contributions from 
Edith Esch, Marie-Jose Gremmo, David 
Little, Harvey Moulden, Philip Riley and 
David Singleton. Dublin: Authentik. 

Little, D. G., and A. J. Grant, 1986: "Learning 
German without a teacher: report on a self- 
mstructional programme for undergraduate 
students of Engineering Science at Trinity 
College, Dublin, 1982-4", CLCS Occasional 
Paper No.l4. 



Learner autonomy 



Little, D., 1991: Lcnrncy Auhvionn/. 7; Definitions, 
Issues niui Problems. Dublin: Authentik. 

Little, D., forthcoming: "Learner autonomy: a 
theoretical construct and its practical 
application". Die Neuercii Sprncheii. 



Little, D. G., and E. J. Davis, 1986: "Interactive 
video for language learnmg: the Autotutor", 
S\istem 14, pp.29-34. 

Little, D., 1994: "Interactive videocasscttc for 
self-access: a preliminary report on the 
implementation of Autotutor 11", Computers 
ill Education, 165-170. 



Definition of language teaching 
sylLabuses 



Little, D., H. 6 Murchu, and D. Singleton, 1985: 
A Functioiwl-Notionnl SyUnbus for Adult 
Leanwrs of Irish. Dublin: Trinity College, 
Centre for Language and Communication 
Studies. 

Little, D., H. 6 Murchu, and D. Suigleton, 1985: 
Towards n Communicntire Curriculum for Irish. 
Dublin: Trinity Collej Centre for 
Language and Communication Studies, in 
association with Bord na Gaeilge. 

Singleton, D. M., and D. G. Little, 1985: "Foreign 
languages at second level: defining a 
syllabus, with particular reference to the 
needs of the senior cycle". CLCS Occa.s'ional 
Paper No.l2. 



Interactive video in language 
learning 



Little, D., 1988: "Interactive video: teaching tool 
or learning resource?", louriud of Applied 
Linguistics 4 (Grool Applied Linguistics 
Association), pp.6i-74. 



Development of language 
learning materials 



Little, D., S. Devitt and D. Singleton, 1989: 
Learning J'oreif;n l.nn^un^es from Authentic 
Texts: Theory and Prnctice. Dublin: Authentik, 
in cissociiition with the Centre for 
Infori ' lion on I anguage Teaching, 
London. 
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Pedagogical grammar 



Singleton, D. M., 1989: Lnii^uagc Acquisition: the 
Age factor. Clcvcdon & Philadelphia: 
Multilingual Matters. 



Little, D., 1994: "Words and their properties: 
arguments for a lexical approach to 
pedagogical grammar." In T. Odlin (ed.). 
Perspectives on Pedngogicnl Grammar, pp.99- 
122. Cambridge: Cambridge University 
Press. 

Little, D., and D. Singleton, 1991: "Authentic 
texts: pedagogic grammar and language 
awareness in foreign language learning". In 
Language Awareness in the Classroom, edited 
by C. James and P. Garrett, pp.123-32. 
London: Longman. 



Second language mental lexicon 

Singleton, D., and D. Little, 1991: "The second 
language lexicon: some evidence from 
university-level learners of French and 
Germar;'', Second Language Research 7, pp.61- 
81. 



The age factor in language 
learning 

Singleton, D. M., 1981: "Age as a factor in 
second language acquisition". CLCS 
Occasional Paper No.3. 



Role of mother tongue and 
language transfer in language 
learning 

Scarpa, F., 1990: "Contrastive analysis and 
second language learners' errors: an 
analysis of C-test data elicited from 
beginners in Italian". CLCS Occasional 
Paper No.27. 

Singleton, D. M., 1981: "Language transfer: a 
review of some recent research". CLCS 
Occasional Paper Nb.l. 

Singleton, D. M., 1987: "Mother and other 
tongue influence on learner French: a case 
study". Studies in Second Language 
Acquisition 9.3. 

Singleton, D. M., 1987: "The fall and rise of 
language transfer". In Tlie Advanced 
Language Learner, edited by J. Coleman and 
R. Towell. London: CILT. 

Singleton, D. M., 1990: "The cross-linguistic 
factor in second language learning: a report 
on some small-scale studies recently 
conducted at the CLCS", CLCS Occasional 
Paper No.24. 

Singleton, D. M., and D. G. Little, 1984: "A first 
encounter with Dutch: perceived language 
distance and language transfer ns factors in 
comprehension". In Language Across 
Cultures, edited by L. Mac Mathuna and D. 
Singleton. Dublin: Irish Association for 
Applied Linguistics. 



Appendix 2 

French module for students of Health Sciences 



Background 



Assessment results 



The French for non-beginners module for 
students in Health Sciences was launched in 
Hilary term a week later than the other 
modules, in order to allow sufficient time for the 
processing of applications. Two-hour contact 
sessions were arranged for Saturday mornings, 
starting at 9 ii^m. The early start meant that both 
this and the French module for students in 
Science (starting at 10 a.m.) could be mnnaged 
by a single team of teachers and language 
assistants. In terms of general design, content, 
organizational structure and assessment 
procedures, the module for students in Health 
Sciences followed a pattern identical to that 
already described. 



Rates of participation 



N'umbor 


Averaj^e 


Hij^host 


Lcnvest 


assessed 


grade 


tirade 


)^rade 


9 


A7'!(. 


82';;, 





Initial 


Number 


Pinal 


Number 


Uital 


allending 


total 


completing 


recruilment 


first session 


recruilment 


module 


39 


18(47':;,) 


39 


9 (23':;,) 



The results indicate a particularly high level nf 
attainment across the students completing the 
module, with an average grade in the upper 
range of the III class, and a first class grade 
achie\'ed by 4 of the 9 students. The difference 
in Ihe level of performance between the students 
in Health Sciences and students in Arts or 
Science was especially salient in the quality of 
the group presentations. This difference is a 
clear reflection of the high standards of 
academic achievement required of those 
entering the Faculty of Health Sciences, many of 
whom have superlative school-leaving results in 
foreign languages. The success demonstrated by 
those completing the French for non-boginners 
module in 1994 points to the desirability of 
making further provision towards language 
Icarnmg opportunities for students in Health 
Sciences, ideally within the same funding 
scheme as that governing the modules for 
students in Arts and Science. 
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Appendix 3 

Summary of expenditure 



ERIC 



staff 

Project co-ordiimtor and pedagogical adviser 27,100 
Hourly paid teachers (3 Trench, 3 German) and 

ERASMUS language assistants (13 French, 17 German) 10,204 



27 



Equipment 

Computer network: 1 server, 10 computers, network cards, 

cabling, printer, furniture, 3 CD-ROM dri\'es 24,549 
Satellite television distribution system, 15 rccei\'ers, 

15 video cassette recorders, cabling, furniture 15,092 

CCTV surveillance system 1,520 

Reconfiguration of language laboratories 1,360 

Computer and printer for project administration 1,912 



Travel and subsistence 

lixternal evaluation 1,585 

Exploratory visits to Amsterdam, Paris and 1-rankfurt 1,742 

EU ROC ALL 1994 1,580 

Third CERCLES Conferem--- ] ,046 



37,304 



44,433 

Language learning materials 5,636 
External evaluation 

Pee 2,078 
Administration coses (Cambridge) 2,613 



4,691 



5,953 

Stationery, printing, photocopying, office consumables 1,9<S3 
TOTAL £100,000 



Appendix 4 — Band descriptors and rating sheet 

Assessment Guidelines (Non-Beginners) 

Expected maximum global level of attainment 

Can speak at length with confidence and reasonable fluency on familiar or general topics, adapting 
appropriately to formal/informal contexts. Lacks facility in handling abstract or unfamiliar topics but can 
cope reasonably well. Difficulty with complex sentence and discourse structures. 



Individual Student Assessment 





T 1 

l_iC V CI JL 


Level 2 


Level 3 


Level 4 


Level 5 


Phonology: 


almost 


strong LI 


rair lz 


gOOCl 


\ ^ry ^tJOLl 1-1— 




unintelligible 


accent 


accent 


accent and 


accent and 










intonation 


intonation 


Fluency: 


very slow 


stilted and 


flows 


fluent, good 


very fluent 




and hesitant 


somewhat 


reasonably 


expression 


and 




- frequent 


hesitant but 


well despite 




expressive. 




stops and 


manages to 


occasional 




normal 




starts 


keep going 


hesitations 




speech 












tempo 


Grammatical control; 


very little 


errors 


good control 


only 


very few 




control - 


frequent, 


of basic 


occasional 


noticeable 




grammatical 


difficulty 


structures. 


errors. 


errors. 




errors 


with 


some errors 


mostly in 


competent 




impede the 


complex 


in more 


control of 


control of 




message 


structures. 


complex 


complex 


complex 






but message 


structures 


structures 


structures 






intelligible 


and 


and 


and 








discourse 


discourse 


discourse 








patterns 


patterns 


patterns 



Group Task Assessment 



How effectively the task is executed in terms of — 




coiitciitlsiilystaucc 


rhetorical skills 


appropriate /n/i,(fi(n(jc 


presentation skills 


vocabulari/ range 


continuitif and flow 


or<^anization 





To be marked out of 10, the presentation must fulfil TWO conditions; 

(a) Students should not read from scripts (prompts, visuals, props, overhead transparencies, etc., may of courbc 
be used); 

(b) The presentation must be interactive (i.e., there should be regular interaction and communication between 
the students in the group, and not just a simple sequencing of individual talks). ^ 
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If the presentation fai 


Is to fulfil ONE of these conditions, it can only be marked out of 8: 
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2 

wc.lk 


3 


4 

s,itts(.utorv 


.5 


6 


7 


8 

MTV j^niid 




















If the presentation fails to fulfil BOTH of these conditions, it can only be marked out of 6: 
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2 


3 


4 

s.ilisl,u liiry 


5 


f) 

R 1 
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Sample rating sheet 



MODULE 


DAY 


GROUP 


ASSESSOR 


Topic: 



Individual student assessment 



Name 


Phonology 


Fluency 


Grammatical control 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 
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3 
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.J 


Final revision 
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Group Task Assessment 



. To be marked out of 10, the presentation must fulfil TWO conditions: 

(a) Students should not read from scripts 

(b) The presentation must be interactive 
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vcrv pooit 
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If the presentation 


ails to fulfil ONE of these conditions, it can only be marked out of 8: 
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2 
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If the presentation 


ails to fulfil BOTH ol 


these conditions, it can only be marked out of 6: 
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Appendix 5 — Correlational analysis of test components 



French Non-Bcginncrs German Non-Beginners German Beginners 





Group 
Presentation 


Dictation 




Group 
Presentation 


Dictation 




Group 
Presentation 


Dictation 


Diclalioi) 


0.425 13'> 




Dictation 


0..17599.1 




Dictation 


0.20419 




C-Tcst 


0.420545 




C-Test 


0.572294 




C-Tcst 


0.117565 


().X05.'i23 


Written 
tost total 


0.458111 




Written 
test total 


0.563065 




Written 
test total 


0.160559 





Communicative test: inter-rater reliability 

1-"()r llic csiimalion ol' intct-nitcr reliability, the Spcaiiiian-Brown Prophecy Formula wa.s used, ba.sed on 
111 nutiiber of independent raters (4) and tir. average corielalion among raters. (For full delaii.s of the 
procedure, sec Henning, G., 1987: A Guide to Language Testing, Cambridge. Ma.ss.: Newbury House, 
pp.82-83) 



French for Non-Beginners Modules: Correlations between raters' marks 



phonology 


rater 1 


rater 2 


rater 1 


fluency 


rater 1 


rater 2 


rater 1 


rater 2 


().47X.W) 






rater 2 


().4.'il849 






rater .1 


0.47427.1 


0..'i.17l.'i1 




rater 1 


0.487429 


0.770671 




rater 4 


0.6.12066 


0.699 1 14 


0.494.'i78 


rater 4 


0..'i9l.'i22 




0.618614 


i!rainin;ir 


rater 1 


rater 2 


rater 1 


group task 


rater 1 


rater 2 


rater 1 


rater 2 


0..'i77794 






rater 2 


0.4864.'i7 






rater .1 


0..'i.iyi27 


0..'i42007 




rater 1 


().7.'i4427 


0..'i9S9().'i 




rater 4 


0..'i.14l24 


O..'i.'i2907 


0..'i02168 


rater 4 


0.619769 


0.1868.'i 


0.4.10811 


total score 


rater 1 


rater 2 


rater 1 






phonology 


0.8.1 


rater 2 


0..'i.'i7626 






Intcr-ralcr 


fluency 


0.85 


rater .1 


0.67692 


0.7S280I 




reliability 


grammar 


0.8.1 


rater 4 


0.7.'i.1l84 


0.7047.'i6 


0.67194.'i 


coefficients 


group task 


0.8.1 














total score 


0.90 


German for Non-Beginners Modules: Correlations between raters' marks 


phonology 


rater 1 


rater 2 


rater 1 


lliicncy 


rater 1 


rater 2 


rater 1 


rater 2 


0.6061.14 






rater 2 


0.4.^4016 






rater .1 


0..'i9.124.1 


()..'i671l.1 




rater 1 


()..'i.1.'i74.'i 


()..'i49166 




rater 4 


0.61171.1 


0.448.'i27 


0..'i91621 


rater 4 


0.116497 


().l8662.'i 


'> 62 1081 


graniniar 


rater 1 


rater 2 


rater 1 


group task 


rater 1 


rater 2 


rater 1 


rater 2 


0.6.'i|4«« 






rater 2 


0..'iO9201 






rater 3 


0.6 1 ."566 1 


0.. "51 24 11 




rater 1 


0..16 1.172 


0.811946 




rater 4 


0..'i076«2 


0.181094 


«..'i8()402 


rater 4 


0.807617 


0.422477 


0.21l.'i.'i6 


total score 


rater 1 


rater 2 


rater 1 






phonology 


0.84 


rater 2 


0.7716.'i 






IntiT-ratcr 


lliicncy 


0.76 


rater .1 


0.682144 


0.7.'i4.'i|2 




reliability 


grammar 


0.8.1 


rater 4 


0.721 IH.-i 


0.49.'i 1 67 


0..'i28646 


eocfTicients 


group task 


0.82 














total seore 


0.89 


German for Beginners Modules: Correlations between raters' marks 


phonology 


rater 1 


rater 2 


rater 1 


lltieney 


rater 1 


rater 2 


rater 1 


rater 2 


0.416709 






rater 2 


0.669049 






rater .1 


0.4.'i267l 


0..'i482 




rater 1 


I).8l110.'i 


0.198949 




rater 4 


0..'i1.'il79 


0.279881 


O.."; .17 289 


rater 4 


I).74l9.'i2 


0.476206 


0.67477 


graiMMiar 


rater 1 


rater 2 


rater 1 


group task 


rater 1 


rater 2 


rater 1 


later 2 


0.44.'i.'i22 






rater 2 


0.700847 






rater 


().771.'i72 


0.167411 




rater 1 


0.841489 


0.92692(1 




rater 4 


0.7477S 


0.1028.'S7 


0.661644 


rater 4 


t).77IO.'i8 


0.971041 


0.886921 


total seore 


rater 1 


rater 2 


rater 1 






phonology 


0.77 


later 2 


tl 77192 






Inter-rater 


lliicncy 


0.87 


rater .1 


(V8I1S7^ 


0.87811 




relinliility 


;.'iaiiiiiiar 


0.81 


rater 4 


0.7881.19 


0.698 1^9 


0 7') 1491 


eoclHcients 


group task 


0.96 














total score 


0.94 
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